Intuitive interaction in a mixed reality system

Tangible physical systems are more intuitive than Intangible virtual Systems. Mixed reality systems are considered as an alternative to virtual systems, bringing advantages of tangible systems into an interaction. However, past research has mainly focussed on technical aspects of incorporating pervasive-ness and immersive-ness in the virtual systems. This paper reports on an empirical study of intuitive Interaction in a Mixed Reality game system for children and the design aspects that could facilitate intuitive Interaction in such systems. A related samples Friedman’s test showed that the Mixed Reality game system demonstrated more intuitive interactions than non-intuitive Interactions. A linear regression analysis further established that the variation in intuitive Interaction in the Mixed Reality system could be statistically significantly explained primarily by physical affordances offered by the Mixed Reality system and to a lesser extent by the perceived affordances in the system. Design guidelines to develop intuitive Mixed Reality systems are discussed. These guidelines should allow designers to exploit the wonders of advances in technology and at the same time allow users to directly interact with the physical real world. This will allow users to access maximal physical affordances, which are primary contributors to intuitive interaction in Tangible and Mixed Reality systems.

SCOOT game event : families as partners in learning

How games can be designed to engage families in learning spaces outside of the classroom. SCOOT Game has been played by families in various science museums and art galleries in Australian capital cities since 2004. Families form groups to collaborate in the game that takes them on an SMS quest through these places engaging them with artworks, historic facts, landmarks, puzzles, street performances etc.

Paper-based mixed reality sketch augmentation as a conceptual design support tool

This undergraduate student paper explores usage of mixed reality techniques as support tools for conceptual design. A proof-of-concept was developed to illustrate this principle. Using this as an example, a small group of designers was interviewed to determine their views on the use of this technology. These interviews are the main contribution of this paper. Several interesting applications were determined, suggesting possible usage in a wide range of domains. Paper-based sketching, mixed reality and sketch augmentation techniques complement each other, and the combination results in a highly intuitive interface.

Mixed reality stories: How creative writers integrate virtual and physical realities

Mixed reality stories (MRS) unfold simultaneously in the physical and the virtual world. Advancements in digital technologies, which are now able to capture more contextual information about our physical environments, are enabling novel ways of blending the two worlds. To explore the process of creating stories from this perspective, we conducted a study with creative writers, in which we asked them to write a MRS script for outdoor running. While we saw instances of intentional connections between physical and virtual worlds in their work, we also observed the use of ambiguity or even deliberate contradiction with available contextual information. In this paper we discuss how these approaches can be beneficial for MRS and propose directions for future work.

Being chased by zombies!: Understanding the experience of mixed reality quests

Both researchers and practitioners show increasing interest in exploring mixed reality games: games, where physical environments blend together with digital technologies. In this paper we have extended earlier work by bringing attention to the role of narrative in mixed reality games. For our case study we chose a mobile phone application Zombies Run!, which is designed to support actual running. This application contains a fictional story about a zombie apocalypse and provides runners with various quests (in the form of missions) to complete during their run. We investigated different aspects of participants' experience with the application and how it changed their running. Our findings show how the app changed running in three major ways. Firstly, it changed the way runs were organised. Secondly, it shook up established running routines. And lastly, it shaped the meanings associated with running.

Immersive interfaces for art education teaching and learning in mixed reality virtual and real world environments

Selected ubiquitous technologies encourage collaborative participation between higher education students and educators within a virtual socially networked e-learning landscape. Multiple modes of teaching and learning, ranging from real world experiences, to text and digital images accessed within the Deakin Studies Online learning management system and a constructed virtual world in which the user’s creative imagination transports them to the “other side” of their computer screens is discussed in this paper. These constructed environments support interaction between communities of learners and enable multiple simultaneous participants to access graphically built 3D environments, interact with digital artifacts and various functional tools and represent themselves through avatars, to communicate with other participants and engage in collaborative art learning. A narrative interpretative research approach was used to profile the 21st century higher education student learner, to investigate the lived experience and multiple art learning perspectives documented in student visual journal entries and art educator observations to ascertain if an e-technology rich augmented learning environment resulted in the establishment of more effective e-learning communities of practice.

The generation of scenes in mixed reality environments using the chromakey technique

A classic problem in the development of Mixed Reality systems is the registration. The correct alignment between virtual objects and the real elements is extremely important for the coherent composition of the resultant scene. Considering this context, this paper describes an approach for the composition of scenes in Mixed Reality environments using the chromakey technique for the extraction of real objects. After that, the scene is mounted in a coherent way related to the depth in OpenGL framebuffer for posterior rendering. ©2007 IEEE.

System Architecture of a Mixed Reality Framework

In this paper the software architecture of a framework which simplifies the development of applications in the area of Virtual and Augmented Reality is presented. It is based on VRML/X3D to enable rendering of audio-visual information. We extended our VRML rendering system by a device management system that is based on the concept of a data-flow graph. The aim of the system is to create Mixed Reality (MR) applications simply by plugging together small prefabricated software components, instead of compiling monolithic C++ applications. The flexibility and the advantages of the presented framework are explained on the basis of an exemplary implementation of a classic Augmented Realityapplication and its extension to a collaborative remote expert scenario.

Increasing Realism and Supporting Content Planning for Dynamic Scenes in a Mixed Reality System incorporating a Time-of-Flight Camera

For broadcasting purposes MIXED REALITY, the combination of real and virtual scene content, has become ubiquitous nowadays. Mixed Reality recording still requires expensive studio setups and is often limited to simple color keying. We present a system for Mixed Reality applications which uses depth keying and provides threedimensional mixing of real and artificial content. It features enhanced realism through automatic shadow computation which we consider a core issue to obtain realism and a convincing visual perception, besides the correct alignment of the two modalities and correct occlusion handling. Furthermore we present a possibility to support placement of virtual content in the scene. Core feature of our system is the incorporation of a TIME-OF-FLIGHT (TOF)-camera device. This device delivers real-time depth images of the environment at a reasonable resolution and quality. This camera is used to build a static environment model and it also allows correct handling of mutual occlusions between real and virtual content, shadow computation and enhanced content planning. The presented system is inexpensive, compact, mobile, flexible and provides convenient calibration procedures. Chroma-keying is replaced by depth-keying which is efficiently performed on the GRAPHICS PROCESSING UNIT (GPU) by the usage of an environment model and the current ToF-camera image. Automatic extraction and tracking of dynamic scene content is herewith performed and this information is used for planning and alignment of virtual content. An additional sustainable feature is that depth maps of the mixed content are available in real-time, which makes the approach suitable for future 3DTV productions. The presented paper gives an overview of the whole system approach including camera calibration, environment model generation, real-time keying and mixing of virtual and real content, shadowing for virtual content and dynamic object tracking for content planning.

The MIRELA framework: modeling and analyzing mixed reality applications using timed automata

Mixed Reality (MR) aims to link virtual entities with the real world and has many applications such as military and medical domains [JBL+00, NFB07]. In many MR systems and more precisely in augmented scenes, one needs the application to render the virtual part accurately at the right time. To achieve this, such systems acquire data related to the real world from a set of sensors before rendering virtual entities. A suitable system architecture should minimize the delays to keep the overall system delay (also called end-to-end latency) within the requirements for real-time performance. In this context, we propose a compositional modeling framework for MR software architectures in order to specify, simulate and validate formally the time constraints of such systems. Our approach is first based on a functional decomposition of such systems into generic components. The obtained elements as well as their typical interactions give rise to generic representations in terms of timed automata. A whole system is then obtained as a composition of such defined components. To write specifications, a textual language named MIRELA (MIxed REality LAnguage) is proposed along with the corresponding compilation tools. The generated output contains timed automata in UPPAAL format for simulation and verification of time constraints. These automata may also be used to generate source code skeletons for an implementation on a MR platform. The approach is illustrated first on a small example. A realistic case study is also developed. It is modeled by several timed automata synchronizing through channels and including a large number of time constraints. Both systems have been simulated in UPPAAL and checked against the required behavioral properties.

Application of Time-Delay Estimation to Mixed Reality Multisensor Tracking

Spatial tracking is one of the most challenging and important parts of Mixed Reality environments. Many applications, especially in the domain of Augmented Reality, rely on the fusion of several tracking systems in order to optimize the overall performance. While the topic of spatial tracking sensor fusion has already seen considerable interest, most results only deal with the integration of carefully arranged setups as opposed to dynamic sensor fusion setups. A crucial prerequisite for correct sensor fusion is the temporal alignment of the tracking data from several sensors. Tracking sensors are typically encountered in Mixed Reality applications, are generally not synchronized. We present a general method to calibrate the temporal offset between different sensors by the Time Delay Estimation method which can be used to perform on-line temporal calibration. By applying Time Delay Estimation on the tracking data, we show that the temporal offset between generic Mixed Reality spatial tracking sensors can be calibrated. To show the correctness and the feasibility of this approach, we have examined different variations of our method and evaluated various combinations of tracking sensors. We furthermore integrated this time synchronization method into our UBITRACK Mixed Reality tracking framework to provide facilities for calibration and real-time data alignment.

Studio e prototipazione di un'architettura software di alto livello per lo sviluppo di applicazioni di mixed reality su piattaforma HoloLens

La tesi si pone come obiettivo quello di realizzare un'architettura di alto livello per lo sviluppo di applicazioni dirette alla piattaforma HoloLens. Per conseguire tale risultato si è rivelata necessaria una prima parte di studio dei concetti di mixed reality, con riferimento particolare al caso specifico HoloLens, per poi dirigere l'attenzione alla comprensione dell'architettura di applicazioni olografiche. L'analisi delle API rilasciate per lo sviluppo di applicazioni HoloLens ha permesso di riscontrare varie criticità, alle quali si è posto rimedio tramite l'introduzione di un livello di astrazione, che possa consentire uno sviluppo di applicazioni ad un livello più alto. Si è poi introdotto il concetto di augmented worlds (mondi aumentati), i cui principi cardine hanno fornito le basi per la progettazione e lo sviluppo di un sistema client-server, nel quale il dispositivo HoloLens agisce come un client e la logica di controllo degli elementi del modello dell'applicazione viene gestita lato server.

Applicazioni di Mixed Reality e Microsoft Hololens: studio e riprogettazione del concetto di ologramma e della relativa gestione all'interno di un'applicazione olografica.

La tesi presenta una panoramica sull'augmented, virtual e mixed reality, descrivendone le caratteristiche e le modalità di sviluppo. Come caso di studio viene analizzato il dispositivo Microsoft Hololens, descrivendone le caratteristiche concettuali, hardware e software. Per le applicazioni di questo dispositivo viene effettuata una riprogettazione della gestione e del concetto di ologramma all'interno di un'applicazione olografica, analizzandone i motivi e i vantaggi. E' fornita una overview sui dettagli implementativi della riprogettazione al fine di chiarire ogni aspetto dell'applicazione.

Ambienti di servizi per applicazioni in mixed reality per MS HoloLens

Nella presente tesi, dopo una prima parte dedicata all'analisi del concetto di realtà mediata e delle sue varie specializzazioni, verranno illustrati i principali dispositivi hardware presenti attualmente sul mercato, analizzandone le caratteristiche, gli aspetti innovativi e la diffusione raggiunta al momento. Si passerà quindi ad esaminare nel dettaglio il visore Microsoft HoloLens, impiegato per la realizzazione del progetto di laurea, per arrivare alla descrizione dello sviluppo di un'applicazione. Il software realizzato rappresenta un esempio di applicazione della realtà mista al campo del design di interni: verranno esaminati strumenti di sviluppo, architettura, casi d'uso e work-flow dell'applicazione. Infine verranno descritte le fasi di test, i costi di sviluppo dell'applicazione e il feedback ricevuto dagli utenti.

Reproduction game demo

An 8 screen stereoscopic 3D Multi-user environment set in a circular formation. Paticipants are tracked by 8 kinect sensors as they move around the 360 degree installation. Participants inherit a species upon entering. Artificial Life based entities cluster towards the human particpants of their species and their songs evolve according to the interactions.